Methods and apparatus for the use of slice groups in encoding multi-view video coding (mvc) information

ABSTRACT

There are provided methods and apparatus for the use of slice groups in encoding and decoding multi-view video coding (MVC) information. An apparatus and method are described which encodes at least two views corresponding to multi-view video content using multiple slice groups. Furthermore, an apparatus and method are described for decoding at least two views corresponding to multi-view video content using multiple slice groups.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. application Ser. No.15/067,300 filed on Mar. 11, 2016, which claims the benefit of U.S.application Ser. No. 12/450,694 filed on Oct. 7, 2009 (U.S. Pat. No.9,313,515), which is a National Phase entry under 35 U.S.C. § 371 ofInternational Application No. PCT/US2008/005047, filed Apr. 18, 2008,which claims benefit of Provisional Application No. 60/938,363, filedMay 16, 2007, which is incorporated by reference herein in its entirety.Further, this application is related to the non-provisional application,Attorney Docket No. PU080024, entitled “METHODS AND APPARATUS FOR THEUSE OF SLICE GROUPS IN DECODING MULTI-VIEW VIDEO CODING (MVC)INFORMATION”, which is commonly assigned, incorporated by referenceherein.

TECHNICAL FIELD

The present principles relate generally to video encoding and decodingand, more particularly, to methods and apparatus for the use of slicegroups in encoding and decoding multi-view video coding (MVC)information.

BACKGROUND

The International Organization for Standardization/InternationalElectrotechnical Commission (ISO/IEC) Moving Picture Experts Group-4(MPEG-4) Part 10 Advanced Video Coding (AVC) standard/InternationalTelecommunication Union, Telecommunication Sector (ITU-T) H.264recommendation (hereinafter the “MPEG-4 AVC standard”) provides aframework for 3D video coding through a Supplemental EnhancementInformation (SEI) message referred to as a Stereo Video Information(SVI) SEI message, as shown in Table 5.

TABLE 5 C Descriptor stereo_video_info( payloadSize ) { field_views_flag5 u(1) if( field_views_flag ) top_field_is_left_view_flag 5 u(1) else {current_frame_is_left_view_flag 5 u(1) next_frame_is_second_view_flag 5u(1) } left_view_self_contained_flag 5 u(1)right_view_self_contained_flag 5 u(1) }

The SVI SEI message provides the decoder with an indication that theentire coded video sequence includes pairs of pictures formingstereo-view content. Further, the SVI SEI message specifies whether thetwo views are arranged in field format or frame format and whether andwhich of those views have been coded independently. Under field format,each view of a given time instance is coded into each field ofinterlaced video frame. Under frame format, each alternative frame inthe output order represents a view.

The MPEG-4 AVC Standard introduced additional high level syntax topartition each picture into slices. Additionally these slices can belongto different slice groups. These slice groups can have different slicegroup types. Multiple slice groups make it possible to map the sequenceof coded macoblocks (MBs) to the decoded picture in a number of flexibleways. The allocation of macroblocks is determined by a macroblock toslice group map that indicates which slice group each macroblock belongsto. The MPEG-4 AVC Standard supports the following different slice grouptypes: interleaved slice groups; dispersed slice group mapping; one ormore “foreground” slice groups and a “leftover” slice group; changingslice groups; and explicit assignment of a slice group to each slicegroup map unit. This provides a lot of flexibility in how the differentslices are coded. The primary purpose of slice groups can be consideredto be error resiliency. Another application for slice groups is regionof interest (ROI) coding.

The MPEG-4 AVC Standard also includes a SEI message that is used tospecify the constraint of inter prediction over slice group boundaries.This can be useful for parallel processing.

SUMMARY

These and other drawbacks and disadvantages of the prior art areaddressed by the present principles, which are directed to methods andapparatus for the use of slice groups in encoding and decodingmulti-view video coding (MVC) information.

According to an aspect of the present principles, there is provided anapparatus. The apparatus includes an encoder for encoding at least twoviews corresponding to multi-view video content using multiple slicegroups.

According to another aspect of the present principles, there is provideda method. The method includes encoding at least two views correspondingto multi-view video content using multiple slice groups.

According to yet another aspect of the present principles, there isprovided an apparatus. The apparatus includes a decoder for decoding atleast two views corresponding to multi-view video content using multipleslice groups.

According to a further aspect of the present principles, there isprovided a method. The method includes decoding at least two viewscorresponding to multi-view video content using multiple slice groups.

These and other aspects, features and advantages of the presentprinciples will become apparent from the following detailed descriptionof exemplary embodiments, which is to be read in connection with theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The present principles may be better understood in accordance with thefollowing exemplary figures, in which:

FIG. 1 is a block diagram for a video encoder capable of performingvideo encoding in accordance with a multi-view video coding (MVC)extension of the MPEG-4 AVC Standard;

FIG. 2 is a block diagram for a video decoder capable of performingvideo decoding in accordance with a multi-view video coding (MVC)extension of the MPEG-4 AVC Standard;

FIG. 3 is a diagram for an inter-view-temporal prediction structurebased on the MPEG-4 AVC standard, using hierarchical B pictures, towhich the present principles may be applied, in accordance with anembodiment of the present principles;

FIG. 4 is a diagram for an exemplary tiling of four views on a singleframe, in accordance with an embodiment of the present principles;

FIG. 5 is a flow diagram for an exemplary method for encoding multi-viewvideo content using slice groups, in accordance with an embodiment ofthe present principles;

FIG. 6 is a flow diagram for an exemplary method for decoding multi-viewvideo content using slice groups, in accordance with an embodiment ofthe present principles;

FIG. 7 is a flow diagram for another exemplary method for encodingmulti-view video content using slice groups, in accordance with anembodiment of the present principles;

FIG. 8 is a flow diagram for another exemplary method for decodingmulti-view video content using slice groups, in accordance with anembodiment of the present principles;

FIG. 9 is a flow diagram for yet another exemplary method for encodingmulti-view video content using slice groups, in accordance with anembodiment of the present principles; and

FIG. 10 is a flow diagram for yet another exemplary method for decodingmulti-view video content using slice groups, in accordance with anembodiment of the present principles.

DETAILED DESCRIPTION

The present principles are directed to methods and apparatus for the useof slice groups in encoding and decoding multi-view video coding (MVC)information.

The present description illustrates the present principles. It will thusbe appreciated that those skilled in the art will be able to devisevarious arrangements that, although not explicitly described or shownherein, embody the present principles and are included within its spiritand scope.

All examples and conditional language recited herein are intended forpedagogical purposes to aid the reader in understanding the presentprinciples and the concepts contributed by the inventor(s) to furtheringthe art, and are to be construed as being without limitation to suchspecifically recited examples and conditions.

Moreover, all statements herein reciting principles, aspects, andembodiments of the present principles, as well as specific examplesthereof, are intended to encompass both structural and functionalequivalents thereof. Additionally, it is intended that such equivalentsinclude both currently known equivalents as well as equivalentsdeveloped in the future, i.e., any elements developed that perform thesame function, regardless of structure.

Thus, for example, it will be appreciated by those skilled in the artthat the block diagrams presented herein represent conceptual views ofillustrative circuitry embodying the present principles. Similarly, itwill be appreciated that any flow charts, flow diagrams, statetransition diagrams, pseudocode, and the like represent variousprocesses which may be substantially represented in computer readablemedia and so executed by a computer or processor, whether or not suchcomputer or processor is explicitly shown.

The functions of the various elements shown in the figures may beprovided through the use of dedicated hardware as well as hardwarecapable of executing software in association with appropriate software.When provided by a processor, the functions may be provided by a singlededicated processor, by a single shared processor, or by a plurality ofindividual processors, some of which may be shared. Moreover, explicituse of the term “processor” or “controller” should not be construed torefer exclusively to hardware capable of executing software, and mayimplicitly include, without limitation, digital signal processor (“DSP”)hardware, read-only memory (“ROM”) for storing software, random accessmemory (“RAM”), and non-volatile storage.

Other hardware, conventional and/or custom, may also be included.Similarly, any switches shown in the figures are conceptual only. Theirfunction may be carried out through the operation of program logic,through dedicated logic, through the interaction of program control anddedicated logic, or even manually, the particular technique beingselectable by the implementer as more specifically understood from thecontext.

In the claims hereof, any element expressed as a means for performing aspecified function is intended to encompass any way of performing thatfunction including, for example, a) a combination of circuit elementsthat performs that function or b) software in any form, including,therefore, firmware, microcode or the like, combined with appropriatecircuitry for executing that software to perform the function. Thepresent principles as defined by such claims reside in the fact that thefunctionalities provided by the various recited means are combined andbrought together in the manner which the claims call for. It is thusregarded that any means that can provide those functionalities areequivalent to those shown herein.

Reference in the specification to “one embodiment” or “an embodiment” ofthe present principles means that a particular feature, structure,characteristic, and so forth described in connection with the embodimentis included in at least one embodiment of the present principles. Thus,the appearances of the phrase “in one embodiment” or “in an embodiment”appearing in various places throughout the specification are notnecessarily all referring to the same embodiment.

It is to be appreciated that the use of the term “and/or”, for example,in the case of “A and/or B”, is intended to encompass the selection ofthe first listed option (A), the selection of the second listed option(B), or the selection of both options (A and B). As a further example,in the case of “A, B, and/or C”, such phrasing is intended to encompassthe selection of the first listed option (A), the selection of thesecond listed option (B), the selection of the third listed option (C),the selection of the first and the second listed options (A and B), theselection of the first and third listed options (A and C), the selectionof the second and third listed options (B and C), or the selection ofall three options (A and B and C). This may be extended, as readilyapparent by one of ordinary skill in this and related arts, for as manyitems listed.

Moreover, it is to be appreciated that while one or more embodiments ofthe present principles are described herein with respect to the MPEG-4AVC standard, the present principles are not limited to solely thisstandard and, thus, may be utilized with respect to other video codingstandards, recommendations, and extensions thereof, including extensionsof the MPEG-4 AVC standard, while maintaining the spirit of the presentprinciples.

As used herein, “high level syntax” refers to syntax present in thebitstream that resides hierarchically above the macroblock layer. Forexample, high level syntax, as used herein, may refer to, but is notlimited to, syntax at the slice header level, Supplemental EnhancementInformation (SEI) level, Picture Parameter Set (PPS) level, SequenceParameter Set (SPS) level and Network Abstraction Layer (NAL) unitheader level.

Turning to FIG. 1, a video encoder capable of performing video encodingin accordance with a multi-view video coding (MVC) extension of theMPEG-4 AVC Standard is indicated generally by the reference numeral 100.

The video encoder 100 includes a frame ordering buffer 110 having anoutput in signal communication with a non-inverting input of a combiner185. An output of the combiner 185 is connected in signal communicationwith a first input of a transformer and quantizer 125. An output of thetransformer and quantizer 125 is connected in signal communication witha first input of an entropy coder 145 and a first input of an inversetransformer and inverse quantizer 150. An output of the entropy coder145 is connected in signal communication with a first non-invertinginput of a combiner 190. An output of the combiner 190 is connected insignal communication with a first input of an output buffer 135.

A first output of an encoder controller 105 is connected in signalcommunication with a second input of the frame ordering buffer 110, asecond input of the inverse transformer and inverse quantizer 150, aninput of a picture-type decision module 115, an input of amacroblock-type (MB-type) decision module 120, a second input of anintra prediction module 160, a second input of a deblocking filter 165,a first input of a motion compensator 170, a first input of a motionestimator 175, and a second input of a reference picture buffer 180.

A second output of the encoder controller 105 is connected in signalcommunication with a first input of a Supplemental EnhancementInformation (SEI) inserter 130, a second input of the transformer andquantizer 125, a second input of the entropy coder 145, a second inputof the output buffer 135, and an input of the Sequence Parameter Set(SPS) and Picture Parameter Set (PPS) inserter 140.

A first output of the picture-type decision module 115 is connected insignal communication with a third input of a frame ordering buffer 110.A second output of the picture-type decision module 115 is connected insignal communication with a second input of a macroblock-type decisionmodule 120.

An output of the Sequence Parameter Set (SPS) and Picture Parameter Set(PPS) inserter 140 is connected in signal communication with a thirdnon-inverting input of the combiner 190.

An output of the inverse quantizer and inverse transformer 150 isconnected in signal communication with a first non-inverting input of acombiner 119. An output of the combiner 119 is connected in signalcommunication with a first input of the intra prediction module 160 anda first input of the deblocking filter 165. An output of the deblockingfilter 165 is connected in signal communication with a first input of areference picture buffer 180. An output of the reference picture buffer180 is connected in signal communication with a second input of themotion estimator 175. A first output of the motion estimator 175 isconnected in signal communication with a second input of the motioncompensator 170. A second output of the motion estimator 175 isconnected in signal communication with a third input of the entropycoder 145.

An output of the motion compensator 170 is connected in signalcommunication with a first input of a switch 197. An output of the intraprediction module 160 is connected in signal communication with a secondinput of the switch 197. An output of the macroblock-type decisionmodule 120 is connected in signal communication with a third input ofthe switch 197. The third input of the switch 197 determines whether ornot the “data” input of the switch (as compared to the control input,i.e., the third input) is to be provided by the motion compensator 170or the intra prediction module 160. The output of the switch 197 isconnected in signal communication with a second non-inverting input ofthe combiner 119 and with an inverting input of the combiner 185.

A second input of the encoder control receives slice group information186. View 0 through View N-1, collectively represented by the referencenumeral 189, provided to a tiling module 188 that tiles the views toprovide an input picture(s) 101.

First inputs of the frame ordering buffer 110 and the encoder controller105 are available as input of the encoder 100, for receiving an inputpicture 101. Moreover, an input of the Supplemental EnhancementInformation (SEI) inserter 130 is available as an input of the encoder100, for receiving metadata. An output of the output buffer 135 isavailable as an output of the encoder 100, for outputting a bitstream.

Turning to FIG. 2, a video decoder capable of performing video decodingin accordance with a multi-view video coding (MVC) extension of theMPEG-4 AVC Standard is indicated generally by the reference numeral 200.

The video decoder 200 includes an input buffer 210 having an outputconnected in signal communication with a first input of the entropydecoder 245. A first output of the entropy decoder 245 is connected insignal communication with a first input of an inverse transformer andinverse quantizer 250. An output of the inverse transformer and inversequantizer 250 is connected in signal communication with a secondnon-inverting input of a combiner 225. An output of the combiner 225 isconnected in signal communication with a second input of a deblockingfilter 265 and a first input of an intra prediction module 260. A secondoutput of the deblocking filter 265 is connected in signal communicationwith a first input of a reference picture buffer 280. An output of thereference picture buffer 280 is connected in signal communication with asecond input of a motion compensator 270.

A second output of the entropy decoder 245 is connected in signalcommunication with a third input of the motion compensator 270 and afirst input of the deblocking filter 265. A third output of the entropydecoder 245 is connected in signal communication with an input of adecoder controller 205. A first output of the decoder controller 205 isconnected in signal communication with a second input of the entropydecoder 245. A second output of the decoder controller 205 is connectedin signal communication with a second input of the inverse transformerand inverse quantizer 250. A third output of the decoder controller 205is connected in signal communication with a third input of thedeblocking filter 265. A fourth output of the decoder controller 205 isconnected in signal communication with a second input of the intraprediction module 260, with a first input of the motion compensator 270,and with a second input of the reference picture buffer 280.

An output of the motion compensator 270 is connected in signalcommunication with a first input of a switch 297. An output of the intraprediction module 260 is connected in signal communication with a secondinput of the switch 297. An output of the switch 297 is connected insignal communication with a first non-inverting input of the combiner225.

An input of the input buffer 210 is available as an input of the decoder200, for receiving an input bitstream. A first output of the deblockingfilter 265 is available as an output of the decoder 200, for outputtingan output picture.

Turning to FIG. 3, an inter-view-temporal prediction structure based onthe MPEG-4 AVC standard, using hierarchical B pictures, is indicatedgenerally by the reference numeral 300.

Turning to FIG. 4, an exemplary tiling of four views on a single frameis indicated generally by the reference numeral 400.

In accordance with the present principles, we extend the MPEG-4 AVCStandard framework to enable the coding and decoding of multi-view videocontent using slice groups. To support this new capability, we havedeveloped a new high level syntax to signal multi-view video informationin slices using the MPEG-4 AVC Standard. As noted above, although one ormore embodiments are described herein with respect to the MPEG-4 AVCStandard, the present principles are not so limited and can be extendedto other video coding techniques.

In accordance with the present principles, slice groups compatible withthe MPEG-4 AVC Standard are used to code multiple views in a singleframe. As previously mentioned, the MPEG-4 AVC Standard introducedadditional high level syntax to partition each picture into slices.Additionally these slices can belong to different slice groups. Theseslice groups can have different slice group types. The MPEG-4 AVCStandard supports the following different slice group types: interleavedslice groups; dispersed slice group mapping; one or more “foreground”slice groups and a “leftover” slice group; changing slice groups; andexplicit assignment of a slice group to each slice group map unit. Thisprovides a lot of flexibility in how the different slices are coded. Theprimary purpose of slice groups can be considered to be errorresiliency. Another application for slice groups is region of interest(ROI) coding. AVC also includes an SEI message that is used to specifythe constraint of inter prediction over slice group boundaries. This canbe useful for parallel processing.

Taking advantage of the fact that different slice groups can be codedand decoded independently using the SEI message and also the concept ofslice groups, we can code each picture from a different view as aseparate slice group. As noted above, FIG. 4 shows an exemplary tiling400 of different views on a single frame. In the framework of the MPEG-4AVC Standard, each tiled view can be considered to be a particular slicegroup. For example, the top-left quadrant may be slice group 1, thetop-right quadrant may be slice group 2, the bottom-left quadrant may beslice group 3, and the bottom-right quadrant may be slice group 4. Onepossible method to achieve this is to use either slice group type maptypes 2 and 6. Slice group type 2 represents “foreground” slice groupsand a “leftover” slice group. In accordance with the present principles,the “foreground” slice groups can be specified as a rectangular regionwith no “leftover” region or with one view as the leftover region. Anyother combination that results in non-overlapping views is alsopossible. Slice group type 6 allows explicitly signaling of themacroblock to slice group mapping in the syntax. Thus, for each view, itis possible to signal which slice group the view belongs to.

After this signaling, each view can then be constrained for interprediction using an SEI message. TABLE 1 shows motion-constrained slicegroup set Supplemental Enhancement Information (SEI) message syntax foruse with the MPEG-4 AVC Standard, in accordance with an embodiment ofthe present principles. This way we are sure that each view predictsfrom its own view only.

TABLE 1 C Descriptor motion_constrained_slice_group_set( payloadSize ) {num_slice_groups_in_set_minus1 5 ue(v) for( i = 0; i <=num_slice_groups_in_set_minus1; i++) slice_group_id[ i ] 5 u(v)exact_sample_value_match_flag 5 u(1) pan_scan_rect_flag 5 u(1) if(pan_scan_rect_flag ) pan_scan_rect_id 5 ue(v) }

The view information from the different slice groups can be extracted ina few different ways. In one embodiment, we can use the SEI message asshown in TABLE 3. TABLE 3 illustrates a Multi-view Video Coding (MVC)Supplemental Enhancement Information (SEI) message for use with MPEG-4AVC Standard compatible slice groups.

TABLE 3 C Descriptor multiview_display_info( payloadSize ) {num_coded_views_minus1 5 ue(v) org_pic_width_in_mbs_minus1 5 ue(v)org_pic_height_in_mbs_minus1 5 ue(v) for( i = 0; i <=num_coded_views_minus1; i++ ) { view_id[ i ] 5 ue(v) slice_group_id[view_id[i] ] 1 u(v) upsample_view_flag[view_id[ i ]] 5 u(1)if(upsample_view_flag[view_id[ i ]]) upsample_filter[view_id[ i ]] 5u(2) if(upsample_fiter[view_id[i]] == 3) { vert_dim[view_id[i]] 5 ue(v)hor_dim[view_id[i]] 5 ue(v) quantizer[view_id[i]] 5 ue(v) for (yuv= 0;yuv< 3; yuv++) { for (y = 0; y < vert_dim[view_id[i]] − 1; y ++) { for(x = 0; x < hor_dim[view_id[i]] − 1; x ++) filter_coeffss[view_id[i]][yuv][y][x] 5 se(v) } } } } }

The portions which describe the location of each view have been removedsince this information is readily available from the PPS. An additionalsyntax, referred to as slice_group_id[view_id[i]], is added. Thisadditional syntax is used to map the slice groups that are signaled inthe PPS to the view_id they correspond to. Thus using this informationone can determine which slice group corresponds to which view_id. Thesemantics of slice_group_id[view_id[i]] are as follows:

-   -   slice_group_id[ view_id[i]] identifies a slice group of the        slice group map unit in raster scan order that corresponds to a        view specified by view_id[i]. The size of the slice_group_id[        view_id[i]] syntax element is        Ceil(Log2(num_slice_groups_minus1+1)) bits. The value of        slice_group_id[ view_id[i]] shall be in the range of 0 to        num_slice_groups_minus1, inclusive.

In another embodiment, we can signal the view information in the PPSalong with the slice group information. An example of this embodiment isindicated in TABLE 2. That is, TABLE 2 shows modified Picture ParameterSet (PPS) RBSP syntax, in accordance with an embodiment of the presentprinciples.

TABLE 2 C Descriptor pic_parameter_set_rbsp( ) { pic_parameter_set_id 1ue(v) seq_parameter_set_id 1 ue(v) entropy_coding_mode_flag 1 u(1)pic_order_present_flag 1 u(1) num_slice_groups_minus1 1 ue(v) if(num_slice_groups_minus1 > 0 ) { slice_group_map_type 1 ue(v) if(slice_group_map_type = = 0 ) for( iGroup = 0; iGroup <=num_slice_groups_minus1; iGroup++ ) run_length_minus1[ iGroup ] 1 ue(v)else if( slice_group_map_type = = 2 ) for( iGroup = 0; iGroup <num_slice_groups_minus1; iGroup++ ) { top_left[ iGroup ] 1 ue(v)bottom_right[ iGroup ] 1 ue(v) view_id[iGroup] 1 ue(v) } else if(slice_group_map_type = = 3 | | slice_group_map_type = = 4 | |slice_group_map_type = = 5 ) { slice_group_change_direction_flag 1 u(1)slice_group_change_rate_minus1 1 ue(v) } else if( slice_group_map_type == 6 ) { pic_size_in_map_units_minus1 1 ue(v) for( i = 0; i <=pic_size_in_map_units_minus1; i++ ) slice_group_id[ i ] 1 u(v) for( i =0; i <= num_slice_groups_minus1; i++ ) view_id[ i ] 1 ue(v) } }num_ref_idx_l0_active_minus1 1 ue(v) num_ref_idx_l1_active_minus1 1ue(v) weighted_pred_flag 1 u(1) weighted_bipred_idc 1 u(2)pic_init_qp_minus26 /* relative to 26 */ 1 se(v) pic_init_qs_minus26 /*relative to 26 */ 1 se(v) chroma_qp_index_offset 1 se(v)deblocking_filter_control_present_flag 1 u(1)constrained_intra_pred_flag 1 u(1) redundant_pic_cnt_present_flag 1 u(1)if( more_rbsp_data( ) ) { transform_8x8_mode_flag 1 u(1)pic_scaling_matrix_present_flag 1 u(1) if(pic_scaling_matrix_present_flag ) for( i = 0; i < 6 + 2*transform_8x8_mode_flag; i++ ) { pic_scaling_list_present_flag[ i ] 1u(1) if( pic_scaling_list_present_flag[ i ] ) if( i < 6 ) scaling_list(ScalingList4x4[ i ], 16, 1 UseDefaultScalingMatrix4x4Flag[ i ] ) elsescaling_list( ScalingList8x8[ i − 6 ], 64, 1UseDefaultScalingMatrix8x8Flag[ i − 6 ] ) }second_chroma_qp_index_offset 1 se(v) } rbsp_trailing_bits( ) 1 }

Here, when the slice group type is 2 we signal the view_id informationin addition to the left and right support. In case the slice group typeis 6, we then provide a mapping from the slice_group_id to view_id. Thisaffords each slice group an associated view_id. This information canthen be used to extract the different views.

Additionally, we have adapted the SEI message by adding a flag calledself_contained_flag[i] which indicates that the view identified byview_id[i] does not depend on any other view for inter-prediction. Theself_contained_flag[i] is shown in TABLE 4, and defined as follows:

-   -   self contained_flag[i] equal to 1 indicates that no inter        prediction operations within the decoding process for the view        identified by view_id[i] part of the coded video picture refer        to reference areas that are part of an area which correspond to        view_id[j] and i is not equal to j. self_contained_flag[i] equal        to 0 indicates that some inter prediction operations within the        decoding process for the view identified by view_id[i] part of        the coded video picture may or may not refer to reference areas        that are part of an area which correspond to view_id[j] and i is        not equal to j.

TABLE 4 C Descriptor multiview_display_info( payloadSize ) {num_coded_views_minus1 5 ue(v) org_pic_width_in_mbs_minus1 5 ue(v)org_pic_height_in_mbs_minus1 5 ue(v) for( i = 0; i <=num_coded_views_minus1; i++ ) { view_id[ i ] 5 ue(v)self_contained_flag[ i ] 1 u(1) num_parts[view_id[ i ]] 5 ue(v) for( j =0; j <= num_parts[i]; j++ ) { flip_dir[view_id[ i ]][ j ] 5 u(2)loc_left_offset[view_id[ i ]] [ j ] 5 ue(v) loc_top_offset[view_id[ i ]][ j ] 5 ue(v) frame_crop_left_offset[view_id[ i ]] [ j ] 5 ue(v)frame_crop_right_offset[view_id[ i ]] [ j ] 5 ue(v)frame_crop_top_offset[view_id[ i ]] [ j ] 5 ue(v)frame_crop_bottom_offset[view_id[ i ]] [ j ] 5 ue(v) }upsample_view_flag[view_id[ i ]] 5 u(1) if(upsample_view_flag[view_id[ i]]) upsample_filter[view_id[ i ]] 5 u(2) if(upsample_fiter[view_id[i]]== 3) { vert_dim[view_id[i]] 5 ue(v) hor_dim[view_id[i]] 5 ue(v)quantizer[view_id[i]] 5 ue(v) for (yuv= 0; yuv< 3; yuv++) { for (y = 0;y < vert_dim[view_id[i]] − 1; y ++) { for (x = 0; x <hor_dim[view_id[i]] − 1; x ++) filter_coeffss[view_id[i]] [yuv][y][x] 5se(v) } } } } }

Turning to FIG. 5, an exemplary method for encoding multi-view videocontent using slice groups is indicated generally by the referencenumeral 500.

The method 500 includes a start block that passes control to a functionblock 510. The function block 510 inputs views 0 to N-1, and passescontrol to a function block 515. The function block 515 tiles the inputviews, and passes control to a function block 520. The function block520 provides slice group information for each tile, and passes controlto a function block 525. The function block 525 uses the slice grouptype 2 or 6, and passes control to a function block 530. The functionblock 530 creates a Supplemental Enhancement Information (SEI) messageto constrain motion between slice groups, and passes control to afunction block 535. The function block 535 creates Sequence ParameterSet (SPS) and Picture Parameter Set (PPS) information including slicegroups, and passes control to a function block 540. The function block540 encodes the tiled video sequence, and passes control to an end block599.

Turning to FIG. 6, an exemplary method for decoding multi-view videocontent using slice groups is indicated generally by the referencenumeral 600.

The method 600 includes a start block 605 that passes control to afunction block 610. The function block 610 parses a motion constrainedSupplemental Enhancement Information (SEI) message and a SequenceParameter Set (SPS), and passes control to a function block 615. Thefunction block 615 parses a Picture Parameter Set (PPS), and passescontrol to a function block 620. The function block 620 parses slicegroup information, and passes control to a function block 625. Thefunction block 625 parses the slice group types 2 or 6, and passescontrol to a function block 630. The function block 630 decodes eachslice group, and passes control to a function block 635. The functionblock 635 extracts each slice group to obtain different views, andpasses control to an end block 699.

Turning to FIG. 7, another exemplary method for encoding multi-viewvideo content using slice groups is indicated generally by the referencenumeral 700.

The method 700 includes a start block 705 that passes control to afunction block 710. The function block 710 inputs views 0 to N-1, andpasses control to a function block 715. The function block 715 tiles theinput views, and passes control to a function block 720. The functionblock 720 provides slice group information for each tile, and passescontrol to a function block 725. The function block 725 uses slice grouptypes 2 or 6, and passes control to a decision block 730. The decisionblock 730 determines whether or not the current slice group type isequal to 2. If so, the control is passed to a function block 735.Otherwise, control is passed to a function block 740.

The function block 735 specifies the view_id for each slice group alongwith the top_left and bottom_right coordinates, and passes control to afunction block 745.

The function block 745 continues the encoding process, and passescontrol to an end block 799.

The function block 740 specifies the view_id for each slice group, andpasses control to the function block 745.

Turning to FIG. 8, another exemplary method for decoding multi-viewvideo content using slice groups is indicated generally by the referencenumeral 800.

The method 800 includes a start block 805 that passes control to afunction block 810. The function block 810 parses a Sequence ParameterSet (SPS), and passes control to a function block 815. The functionblock 815 parses a Picture Parameter Set (PPS), and passes control to afunction block 820. The function block 820 parses slice groupinformation, and passes control to a decision block 825. The decisionblock 825 determines whether or not the current slice group type isequal to type 2. If so, then control is passed to a function block 830.Otherwise, control is passed to a function block 835.

The function block 830 parses the view_id for each slice group alongwith the top_left and bottom_right coordinates, and passes control to afunction block 840.

The function block 840 continues the decoding process, and passescontrol to an end block 899.

The function block 835 parses the view_id for each slice group, andpasses control to the function block 840.

Turning to FIG. 9, yet another exemplary method for encoding multi-viewvideo content using slice groups is indicated generally by the referencenumeral 900.

The method 900 includes a start block 905 that passes control to afunction block 910. The function block 910 starts a multi-view videocoding Supplemental Enhancement Information (SEI) message, and passescontrol to a function block 915. The function block 915 writes thenumber of coded views minus 1, and passes control to a function block920. The function block 920 writes the original picture width andheight, and passes control to a decision block 925. The decision block925 determines whether or not all views are done. If so, then control ispassed to a function block 930. Otherwise, control is passed to afunction block 935.

The function block 930 encodes the views, and passes control to an endblock 999.

The function block 935 writes the view_id in the SupplementalEnhancement Information (SEI) message, and passes control to a functionblock 940. The function block 940 writes the slice_group_id, and passescontrol to a function block 945. The function block 945 writes theupsample_view_flag in the Supplemental Enhancement Information (SEI)message, and passes control to a function block 950. The function block950 writes other Supplemental Enhancement Information (SEI) parametersin the Supplemental Enhancement Information (SEI) message, and returnscontrol to the decision block 925.

Turning to FIG. 10, yet another exemplary method for decoding multi-viewvideo content using slice groups is indicated generally by the referencenumeral 1000.

The method 1000 includes a start block 1005 that passes control to afunction block 1010. The function block 1010 parses a multi-view videoSupplemental Enhancement Information (SEI) message, and passes controlto a function block 1015. The function block 1015 parses the number ofcoded views minus 1, and passes control to a function block 1020. Thefunction block 1020 parses the original picture width and height, andpasses control to a decision block 1025. The decision block 1025determines whether or not all views are done. If so, the control ispassed to a function block 1030. Otherwise, control is passed to afunction block 1035.

The function block 1030 encodes the views, and passes control to an endblock 1099.

The function block 1035 parses the view_id, and passes control to afunction block 1040. The function block 1040 parses the slice_group_id,and passes control to a function block 1045. The function block 1045parses the upsample_view_flag, and passes control to a function block1050. The function block 1050 parses other Supplemental EnhancementInformation (SEI) parameters, and returns control to the decision block1025.

A description will now be given of some of the many attendantadvantages/features of the present invention, some of which have beenmentioned above. For example, one advantage/feature is an apparatushaving an encoder for encoding at least two views corresponding tomulti-view video content using multiple slice groups.

Another advantage/feature is the apparatus having the encoder asdescribed above, wherein at least some of the multiple slice groups arecompliant with an existing video coding standard or an existing videocoding recommendation.

Yet another advantage/feature is the apparatus having the encoder asdescribed above, wherein at least some of the multiple slice groups arecompliant with the International Organization forStandardization/International Electrotechnical Commission Moving PictureExperts Group-4 Part 10 Advanced Video Coding standard/InternationalTelecommunication Union, Telecommunication Sector H.264 recommendation.

Still another advantage/feature is the apparatus having the encoder asdescribed above, wherein the encoder codes each of the at least twoviews as a respective separate one of the multiple slice groups.

Moreover, another advantage/feature is the apparatus having the encoderthat codes each of the at least two views as a respective separate oneof the multiple slice groups as described above, wherein the encoderuses a Supplemental Enhancement Information message to code each of theat least two views as the respective separate one of the multiple slicegroups.

Further, another advantage/feature is the apparatus having the encoderthat uses the Supplemental Enhancement Information (SEI) message asdescribed above, wherein the Supplemental Enhancement Informationmessage indicates that at least some of the multiple slice groups aremotion constrained.

Also, another advantage/feature is the apparatus having the encoder asdescribed above, wherein the encoder modifies an existing SupplementalEnhancement Information message of an existing video coding standard orvideo coding recommendation to include slice group identifierinformation.

Additionally, another advantage/feature is the apparatus having theencoder as described above, wherein the encoder modifies a PictureParameter Set to signal view identifier information and slice groupidentifier information for the at least two views.

Moreover, another advantage/feature is the apparatus having the encoderas described above, wherein the Picture Parameter Set is modified tospecify at least one of a slice group type 2 and a slice group type 6.

Further, another advantage/feature is the apparatus having the encoderwherein the Picture Parameter Set is modified as described above,wherein a mapping is provided from the view identifier information tothe slice group identifier information, when the slice group type 2 andthe slice group type 6 is specified in the Picture Parameter Set.

Also, another advantage/feature is the apparatus having the encoder asdescribed above, wherein the encoder signals that each of the at leasttwo views is independently coded, using a high level syntax element.

Additionally, another advantage/feature is the apparatus having theencoder as described above, wherein the high level syntax elementcorresponds to at least at one of a slice header level, a sequenceparameter set level, a picture parameter set level, a view parameter setlevel, a network abstraction layer unit header level, and a levelcorresponding to a supplemental enhancement information message.

These and other features and advantages of the present principles may bereadily ascertained by one of ordinary skill in the pertinent art basedon the teachings herein. It is to be understood that the teachings ofthe present principles may be implemented in various forms of hardware,software, firmware, special purpose processors, or combinations thereof.

Most preferably, the teachings of the present principles are implementedas a combination of hardware and software. Moreover, the software may beimplemented as an application program tangibly embodied on a programstorage unit. The application program may be uploaded to, and executedby, a machine comprising any suitable architecture. Preferably, themachine is implemented on a computer platform having hardware such asone or more central processing units (“CPU”), a random access memory(“RAM”), and input/output (“I/O”) interfaces. The computer platform mayalso include an operating system and microinstruction code. The variousprocesses and functions described herein may be either part of themicroinstruction code or part of the application program, or anycombination thereof, which may be executed by a CPU. In addition,various other peripheral units may be connected to the computer platformsuch as an additional data storage unit and a printing unit.

It is to be further understood that, because some of the constituentsystem components and methods depicted in the accompanying drawings arepreferably implemented in software, the actual connections between thesystem components or the process function blocks may differ dependingupon the manner in which the present principles are programmed. Giventhe teachings herein, one of ordinary skill in the pertinent art will beable to contemplate these and similar implementations or configurationsof the present principles.

Although the illustrative embodiments have been described herein withreference to the accompanying drawings, it is to be understood that thepresent principles is not limited to those precise embodiments, and thatvarious changes and modifications may be effected therein by one ofordinary skill in the pertinent art without departing from the scope orspirit of the present principles. All such changes and modifications areintended to be included within the scope of the present principles asset forth in the appended claims.

1. A transmitting method comprising: transmitting at least two viewscorresponding to multi-view video content encoded into a single pictureusing multiple slice groups, wherein the at least two views correspondto a same or similar scene, and wherein each of the at least two viewsis coded as a respective separate one of the multiple slice groups. 2.The method of claim 1, wherein transmitting comprises transmitting aSupplemental Enhancement Information message used to code each of the atleast two views as the respective separate one of the multiple slicegroups.
 3. The method of claim 2, wherein the Supplemental EnhancementInformation message indicates that at least some of the multiple slicegroups are motion constrained.
 4. The method of claim 1, whereintransmitting comprises transmitting a modified version of an existingSupplemental Enhancement Information message of an existing video codingstandard or video coding recommendation, the existing SupplementalEnhancement Information message being modified to include slice groupidentifier information.
 5. The method of claim 1, wherein transmittingcomprises transmitting a modified version of a Picture Parameter Set,the Picture Parameter Set being modified to signal view identifierinformation and slice group identifier information for the at least twoviews.
 6. The method of claim 1, wherein transmitting comprisessignaling that each of the at least two views is independently coded,using a high level syntax element.
 7. A receiving method, comprising:receiving encoded video signal data comprising at least two viewscorresponding to multi-view video content encoded into a single pictureusing multiple slice groups, wherein the at least two views correspondto a same or similar scene, and wherein each of the at least two viewsis coded as a respective separate one of the multiple slice groups. 8.The method of claim 7, wherein receiving comprises receiving aSupplemental Enhancement Information message used to code each of the atleast two views as the respective separate one of the multiple slicegroups.
 9. The method of claim 8, wherein the Supplemental EnhancementInformation message indicates that at least some of the multiple slicegroups are motion constrained.
 10. The method of claim 7, whereinreceiving comprises receiving a modified version of an existingSupplemental Enhancement Information message of an existing video codingstandard or video coding recommendation, the existing SupplementalEnhancement Information message modified to include slice groupidentifier information.
 11. The method of claim 7, wherein receivingcomprises receiving a modified version of a Picture Parameter Set, thePicture Parameter Set modified to signal view identifier information andslice group identifier information for the at least two views.
 12. Themethod of claim 7, wherein receiving comprises determining that each ofthe at least two views is independently coded, using a high level syntaxelement.
 13. A machine readable medium having stored thereon machineexecutable instructions that, when executed, implement an encodingmethod, the encoding method comprising: encoding, into a single picture,at least two views corresponding to multi-view video content usingmultiple slice groups, wherein the at least two views correspond to asame or similar scene, and wherein encoding comprises encoding each ofthe at least two views is coded as a respective separate one of themultiple slice groups.
 14. The medium of claim 13, wherein encodingcomprises encoding a Supplemental Enhancement Information message usedto code each of the at least two views as the respective separate one ofthe multiple slice groups.
 15. The medium of claim 14, wherein theSupplemental Enhancement Information message indicates that at leastsome of the multiple slice groups are motion constrained.
 16. The mediumof claim 13, wherein encoding comprises encoding a modified version ofan existing Supplemental Enhancement Information message of an existingvideo coding standard or video coding recommendation, the existingSupplemental Enhancement Information message being modified to includeslice group identifier information.
 17. The medium of claim 13, whereinencoding comprises encoding a modified version of a Picture ParameterSet, the Picture Parameter Set being modified to signal view identifierinformation and slice group identifier information for the at least twoviews.
 18. The medium of claim 13, wherein encoding comprises signalingthat each of the at least two views is independently coded, using a highlevel syntax element.
 19. A machine readable medium having storedthereon machine executable instructions that, when executed, implement adecoding method, the decoding method comprising: decoding, from a singlepicture, at least two views corresponding to multi-view video contentusing multiple slice groups, wherein the at least two views correspondto a same or similar scene, and wherein decoding comprises decoding eachof the at least two views as a respective separate one of the multipleslice groups.
 20. The medium of claim 19, wherein decoding comprisesdecoding a Supplemental Enhancement Information message used to codeeach of the at least two views as the respective separate one of themultiple slice groups.
 21. The medium of claim 20, wherein theSupplemental Enhancement Information message indicates that at leastsome of the multiple slice groups are motion constrained.
 22. The mediumof claim 19, wherein decoding comprises decoding a modified version ofan existing Supplemental Enhancement Information message of an existingvideo coding standard or video coding recommendation, the existingSupplemental Enhancement Information message modified to include slicegroup identifier information.
 23. The medium of claim 19, whereindecoding comprises decoding a modified version of a Picture ParameterSet, the Picture Parameter Set modified to signal view identifierinformation and slice group identifier information for the at least twoviews.
 24. The medium of claim 19, wherein decoding comprisesdetermining that each of the at least two views is independently coded,using a high level syntax element.